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ABSTRACT 

Market research, important for professional renewal, 
is a critical part of continuing education program planning. It 
includes attention to learning needs and learner motivation to 
participate in continuing education. Informal market research makes 
use of internal sources, such as enrollment data, program 
evaluations, policy statements, and staff members and volunteers; and 
external sources, such as other program providers, professional 
development activities, and market research literature. Formal market 
research with individuals includes surveys, the Delphi technique, 
face-to-face interviews, and telephone interviews. Group research 
techniques include the nominal group and the focus group. Philosophy 
of education, personal preferences, and the work environment 
influence narket research. The steps in executing market research are 
as follows: (1) decide what to research; (2) consider the resources 
available; (3) determine the best time to do research; (4) choose a 
market research approach; (5) recruit people to help; (6) develop a 
preliminary plan for the research; (7) develop a budget; (8) plan how 
to use the data; (9) collect and analyze :he data; and (10) use the 
data to make decisions. (Thirteen references and 33 annotated 
resources for further study are listed, including 3 in marketing, 4 
in market research in continuing education, 19 in market research 
methods, and 7 in research skills.) (CML) 
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Foreword 



In recent years, those planning continuing education programs have 
become more concerned with why individuals participate than in what their 
educational needs might be. Competition among sponsors is one of several 
reasons for this change in interest. 

M. Donald Campbell has combined his experience as a successful 
program planner with a careful examination on proven market research 
methods to provide an exceptional guide to this importantaspectof pro-am 
planning. He offers the reader a number of options to consider when 
gathering information about potential interest in an activity is necessary. 

Along with clear descriptions of different approaches, Dr, Campbell 
analyzes their strengths and weaknesses and strongly emphasizes the role 
planners must play if this aspect of the planning process is to be an effective 
one. Equally important, he clearly reminds all of us that returning to analyze 
the various audiences we serve and our programs is a necessary step if we 
are to avoid dangerous complacency about the quality of our programs and 
the stability of participation, 

Charles E, Kozoll 
Editor 

Guide Series 
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Introduction 



You have just been hired to develop continuing professional education 
programs for middle managers. Ycu have limited work experience, but you 
know that middle managers often want to improve their communication 
skills. You plan to develop a series of workshops on such topics as active 
listening, professional writing, and public presentations, which would be 
part of a certificate program. You can hardly wait to begin but are not sure 
how to proceed. You wonder whether to focus these workshops at a 
beginning or an advanced level, how many workshops to offer, where they 
should be neld, how long they should last, and how much they should cost. 

Or ycu have just accepted responsibility for arts and humanities pro- 
grams in a university continuing education division. You have a strong 
liberal arts background and extensive experience in planning educational 
programs. You are pleased with this new opportunity, because you believe 
many adults, middle-aged and older, would like to increase their under- 
standing of the liberal arts. You would like to develop programs that 
examine contemporary social problems at the local, national, and interna- 
tional levels from a liberal arts perspective. You wonder, though, who would 
participate. Could some programs consider problems that certain profes- 
sionals confront in their practice? Or should these programs appeal more 
broadly to the general public? What social problems would participants 
want to investigate? How much would they pay for these prograins? 

Or you are an experienced program planner, regularly offering a variety 
of annual conferences for health piofcssionals. Working with experts from 
various disciplines, you have planned conferences on family practice 
medicine, health promotion, problems of aging, health care management, 
and other health-related issues for several years. These conferences have 
been very successful. Enrollments have been high, participants have appre- 
ciated the sessions, and revenue has been more than adequate. You are proud 
ofyouraccomplishments, but have lost some enthusiasm for these annual 
events. Planning has become routine, and you wonder if the conferences are 
challenging theparticipantstochangeanypartof their professional practice. 
Conference attendance is still good, but you are concerned that enrollments 
have not been increasing of late. 

Whether you are new to the job or a veteran program planner, market 
research is a critical part of program planning. Regardless of your experi- 
ence or previous success, researching what will draw adults to a contmuing 
education program is an important first step. Studying yuur market before 
plunging into planning can help answer your questions about how tc 
proceed. Even if you lliink you know exactly what your audience needs, 
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2 Building Participation through Market Research 



these adults must share your enthusiasm. If they do not, you and your 
instructor may say hello to an empty room when it is time for the program 
to begin. 

You may have a ^rack record of successful programs, but ongoing 
research is important to guide your planning. Although adult kjamers have 
found your programs useful in the past and continue to participate in them 
now, there is no guarantee they will attend next year or the year after that. 
Program quality may decline slightly. The change may not be noticeable, 
and the participants may still be learning. If thedeclinecontinuss, however, 
attendance may drop off significantly. On the otiier hand, program quality 
may remain very high, but the clientele may change. Change occurs 
continually, whether in a particular profession, a local community, or 
society at large. If year after year programs are based on needs you 
documented years ago, sooner or later these programs will become irrele- 
vant. 

Market research is also important for your own professional renewal. 
Program planning is often exciting, but it can easily become dull. Being a 
step or two removed from the participants is an occupational hazard of 
program planning. Planners do not face the same direct challenge or gain the 
same stimulation from the learners as the instructors do. Unless you make 
an extra effort to interact with the learners, you may not know why they have 
come or what they lake with them. Given the myriad details endemic to the 
life of a program planner, you may be so encumbered by the ABCD's of 
program planning (administration, brochures, budgets, coffee, and donuts) 
that you rarely focus on the E (education). After a while, you may lose 
interest in the programs. Taking time to reflect on what is best for the 
learners is an important way to renew interest in and commitment to 
program planning. 

We will discuss why understanding your clientele has become so 
important and then present both informal approaches and more formal 
market research methods. For each formal method, we will describe the 
method briefly, list its advantages and disadvantages for continuing educa- 
tion market research, and offer key points to consider when using the 
method. We cannot discuss each method thoroughly , but readers who want 
more detailed information are referred to additional references. Then we 
will discuss how you, the program planner, play a critical role in the process 
and how you can proceed with marketresearch, in addition to, or perhaps in 
spite of, all your other responsibilities. 

This publication is not a definitive discussion of market researc h. We will 
not describe all dimensions of marketing. Instead, we will show how a 
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marketing orientation differs in some respects from more traditional ap- 
proaches to needs assessment. Readers who want an in-depth treatment of 
marketing in continuing education should consult references listed at the 
end of this publication. 
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Chapter 1 » . ^ - ■■. 
What is Market Research? 

For the last two decades at least, program planning approaches in 
continuing education have stressed the needs of the learners. An early step 
in these approaches is to discover what the adult needs or wants to learn. 
Then the program is developed to meet these needs and wants. 

Learning Needs 

These program planning approaches are based on Ralph Tyler's (1949) 
principles of curriculum development. Although Tyler was writing for 
elementary and secondary school educators, his principles have blrongly 
influenced continuing education as well. He argues for studying the learners 
to determine the changes needed and then developing educational objec- 
tives designed to produce these changes. 

Applying Tyler's principle^ to continuing educadoiiisnotasslraightfor- 
ward as it may seem. One reason is a different clienteL. Tyler had a captive 
audience in mind: elementary and secondar> school students. They marched 
into thebuilding, whether they liked it or not. The challenge for the educator 
was what and how to teach these bright, shining faces. 

Educators of adults face the same questions, but they have an additional 
challenge. Because participation in most continuing education programs i6 
voluntary, the program planner must first determine how to attract adu lti> to 
a learning experience. Simply focusing on the needs of the learner is not 
enough. The program planner mubt also consider what will motivate adultb 
to participate. Needs can be well documented and good programs devel- 
oped, but adults may not enroll for a variety of reasons. 

A second reason Tyler's principles are problematic for continuing 
education is that social changes have occurred during the past two decades. 
His curriculum development principles were published right after World 
War II, as both our economy and the si^e of our families were expanding. 
The expansion and economic prosperity continued through the 1950s and 
1960s. During these decades we felt, as a nation, that we could afford to do 
almost anything, 

Atthe same time, education was expanding at all levels. First, elementary 
and secondary schools proliferated to accommodate the children bom after 
the war. Then in the 1960s and the first part of the 1970s, higher education 
expanded to serve the "baby boom" generation that was ready for college. 
Girded by an expanding economy, local communities and slate legislatures 
routinely approved new schools and universities. 
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Adult education also cxf nded during these decades, buoyed up by this 
same "can do" spirit. If a new program was needed, we could develop it. Not 
Surprisingly, adult cdu*.ation literature produced in this period focused on 
the needs of the learners. 

By the end of the 1970s, however, the tide had turned. We realized there 
were limits to what we could do. We could not develop a Great Society at 
home and fight an undeclared war in Asia at the same time. Then we 
wondered if wc were depleting our energy resuurces. As energy prices 
soared and a severe economic recession followed, wc a^Kcd if American 
business had lost its competitive edge. We began to think less about the 
needs wc could meet and more about tht resources we had available. 

Changes in education paralleled these social changes. As less money was 
available for education from federal, slate, and local governments, we began 
to think more about what we could accomplish w iih the resources available. 
Just because a new program was needed did not mean that a school or 
university could develop it. 

A iJiird reason that makes Tyler's principles difficult to apply tocontinu- 
ing education programs is the proliferation of providers. Thii> expansion in 
continuing education has paralleled contraction in other educational insti- 
tutions. The baby boom generation is leaving college now and needs to 
continue learning. Two or three decades ago, continuing education was 
primarily the province of land grant universities. Now virtually every 
research and comprehensive university, private college, and community 
college has some kind of continuing education program. Major employers 
have increased their commitment to education and training. Community 
organizations have expanded their lifelong learning programs, and private 
entrepreneurs, offering seminars on a variety ol topics, dot the landscape. 

Since providers of continuing education abound, we should not feel we 
have to meet every adult learning need. We must consider whether a 
potential program fits the mission of our organization and is considered a 
priorit) . We cannot be *'all things to all people." Instead, wc must find our 
programming niche and develop it well. 

Less emphasis, then, is placed on the needs of the learners as the primary 
basis for program planning for three reasons. First, adults usually are not a 
captive audience. Tlic> typically participate voluntarily !.n continuing 
cduLaiion programs. Wc must consider not onl> w hat they need to learn, but 
also what will motivate them to attend an educational program. Second, 
social and economic changes have made us more conscious of resource 
lim itations. Third, pro v idcrs of continuing education arc m uc h more numer- 
ous now than three or four decades ago. Just because a program is needed 
does not mean that our organization can or should provide it. 
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Motivation and Resources 

Carol Aslanian (1985, p. 41) discusses ihcsc same factors. Focusing on 
higher education, she notes that 

the literature on market assessment in adult education is replete with 

surveys that assess need But times have changed. Limited public 

funds, the proliferation of self-supporling campus adult programs. . . 
and increased competition require colleges to examine more than just 
"inlcrest'' . . . Surveys of adult learning that deal with "need" or 
"inlercst" have yielded results that grossly overestimate what adults 
actually will do. This is probably because needing or being interested 
in something does not automatically mean that adults will study 
something. 

We do not mean that the needs of adult learners are irrelevant and should 
be ignored. Needs alone, however, are an insufficient'basis for program 
planning. We must also understand what will motivate adults \o participate 
in educational programs and what resources Ihcy hav " to do so. 

Our approach to market research is based on A.,lanian's framework of 
needs, motivation, and resources. Examining what adults need to learn is an 
important acti vi ty, but we cannot stop there. To develop successful continu- 
ing education programs, we m ust try to understand what will motivate adults 
to participate in our programs. We must also try to determine the resources 
they have available, notably money and time, which enable them to cross the 
threshold and enroll. We do not want our programs priced beyond their 
means. Nor do we want them offered at a time, in a format, or at a location 
that is inaccessible to them. 

Admittedly, incorporating motivation and resources into market re- 
search is much more difficult than simply reporting learning needs. Perhaps 
that is why we often limit our studies to needs assessments. As much as 
possible, however, given the constraints we all face, we must go beyond 
what adults say they need to learn. We should always try to envision a 
successful program and ask orirselves what will make it so. 

Ultimately, the purpose of market research is to predict what adult 
learners will do. The time-honored approaches to assessing needs are an 
important component. But market research should also include efforts to 
determine whether adults will do something about their needs. Will they 
actually enroll in a course? 

Program Planner^s Contribution 

Market research is often perceived strictly as a formal data collection 
process. We may think of conducting a formal research project to document 
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adult learning needs and then feeding that infonr.auon, somewhat mechani- 
cally, into a program planning exercise. Sometimes a report, or even just the 
raw data, can sit indefinitely in a crowded office comer and never influence 
program planning. 

You, the program planner, however, arc the critical ingredient in market 
research. In addition to any formal market research, you should also gather 
information informally to help generate these program ideas. When you do 
conduct formal market research, you must interpret this information. Since 
you are responsible for program planning, you must judge whether the need 
is important and whether your organization should address it. If you do 
proceed with planning, keep in mind the other components of market 
research: learner motivation and learner resources. 

We will describe ways of doing both informal and formal market 
research. Throughout this discussion keep in mind your role as program 
planner. The final section (chapter 5) discusses how to proceed with market 
research, showing how your personal strengths and weaknesses, your own 
philosophy of education, and your working environment all influence your 
approach. 
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Chapter 2 - - .... ^ ..... , 

Informal Market Research 

Importam market informalion is often obt^ned informally. You are 
gathering this in formation all the lime, whether you realize it or not. You are 
influenced by your own organization's history of continuing education 
programs and by the desires and expectations of colleagues. You become 
aware of programs that other organizations offer. And you begin to form 
programideas whilereadingandreflectingon social changes. Thesesources 
of information can all be used more consciously for market research. 

Internal Sources 

Enrollment Data 

Within your own organization, a frequent informal approach is to rev lew 
past pTOgrams. This rev icw is easily done, but typically we Tail to take full 
advanUjgc of the data these internal records provide. We record program 
enrollment and financial results, but often do not lake time to study il^^em. 

Examples of questions to ask as you review enrollment data are: 

1. Do the programs attract the audiences for whom Jiey are intended? 

2. What professional groups are represented? 

3 Do interdisciplinary programs attract mixed audiences, or does one 
profession account for 90 percent of the participants? 

4. Whatage groups arc represented? Do you lend lo attract one age group 
more than others? 

5. How many people have participated in more than oneprogram during 
the past two years? 

6. What communities arc represented? Are the participants primarily 
from the local community? Do virtually all of them live within an 
hour*s drive of the program site? Do they come from all over the 
counuy or primarily from your region? 

For a more detailed discussion of reviewing existing records, sec Falk 
and Miller (1986, pp. 25-27). 

This often overlooked informalion is important for understanding who 
will actually come lo the programs. You know thai these people had 
identified a learning need ihat the program was designed lo meet. More 
importantly, though, they did something aboul that need. They were 
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motivated U) attend, obtained ihe resources to do so, and o vercam e any other 
potential barrier to participation. 

This review of internal market data can guide your program planning. If 
the analysis indicates that the programs for schoolteachers are serving 
exactly whom you want to serve, you know you are on track. If the review 
shows, however, that the programs are drawing primarily mid-career 
teachers, you may want to ^xus more programs on their needs. Or you may 
decide to work harder at attracting less experienced teachers. If administra- 
tors are not enrolling, you may need to change the program content or format 
to attract that group. Do not think, though, that everyone can be served. 
Understand who your participants are, and make sure you meet ilicir needs. 
Then consider serving others. 

Program Evaluations 

Another source of ir mal market data is program evaluations. At the 
end of virtually all contin g education programs, participants are asked to 
complete an evaluation form before they leave or to mail it back soon after 
returning home. Typicall> these forms ask participants to rate the content of 
the program, the instructor, the formal, and the facilities. While reviewing 
these evaluations after each program, you arc forming impressions that 
guide future planning effoi^. If the learners were dissatisfied with the 
program, you know it did not meet their perceived learning needs. If you 
want them to return to future programs, you will have to plan differently. 

Evaluation forms can suggest ideas for future programs. Ask participants 
what other programs they believe are needed and would attend. Ask them 
what areas they would like to examine in more detail ar.d what other topics 
interest th em. Consider these ideas now , but do not put them away and forget 
about them. Be sure to review the data again before beginning your next 
planning phase. 

Policy Statements 

Other sources of information exist within organization. Every organi- 
zation has some kindof mission statement lhatdescribes its purpose. Annual 
reports, long-range planning documents, policy papers, and other docu- 
ments provide a framework for the continuing education programs of your 
organization. These documents address the clientele to be served, content to 
be considered, and instructorqualifications. A university, for example, may 
view continuing education as an extension of its academic programs and 
limit offerings to those that its own faculty can provide. Another university 
may emphasize responding to identified needs by hiring ad hoc faculty to 
address these needs. 
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These policy statements may also discuss whether the continuing educa- 
tion operation should be fully self-supporting or subsidized in some way. 
The organization's policy on self-support 'U influence the fees you must 
charge, w hich in turn influences the audience you can attract For example, 
programs for social workers and human services professionals may need 
subsidizing, because these professionals have lower incomes and their 
employers have limited professional' development funds. Programs for 
physicians, lawyers, and engineer?, or* the other hand, may not be hurt by 
high fees. 

These documents establish the boundaries for the continuing education 
programs you will offer. Do not attempt to offer every program that is 
needed or of interest to you. Some programs, no matter how exciting or how 
necessary, will be intended for a clientele your organization does not serve 
or will include content your organization lacks the instructional resources 
to address. Although "in practice, attention to local agency objectives and 
traditions as a source of new program ideas tends to be implicit and 
informal" (Nowlen, 1980, p. 19), we urge more intenuonal use of these 
materials to help set program directions. These documents obviously are 
important fora new program planner to study, but periodically they provide 
a helpful review for an experienced planner as well. 

Staff Members and Volunteers 

People within the organization are another source of informal market 
information. Superiors, peers, and subordinates, as well as individuals 
serving on the organization's advisory boards, all have a perspective to 
offer. Use every opportunity to interact with these people to do informal 
market research. Their understanding of the organization's history is 
important, especially fora new program planner. Their vision for the future" 
can keep you from getting into a rut. These colleagues are also potential 
sot?rces of new program ideas or excellent reactors to new ideas. 

Working with commiUees is anothei way to gain information from staff 
and volunteers. Standing committees have ongoing responsibilities, but 
Jiey may agree to offer program ideas as part of a regular meeting. Ad hoc 
commiUees can be organized to investigate new program initiatives and 
then participate in planning these programs. For a guide to working with 
planning commitees, see Kozoll (1980). 

External Sources 

Other Program Providers 

Outside your organization, study the programs other providers are 
offering. Review their brochures and other promotional materials. Many 
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already send you ihcir catalogs and brochures. Ask others to add your name 
10 their mailing lists. Lookat brochures displayed at conferences. Talk with 
your colleagues at these conferences about their successes and failures. 
These promotional materials suggest ideas for entire programs, individual 
sessions, and faculty members who can address specific topics. 

Analyze this information carefully. Just because another provider offers 
a program does not mean you should. Your organization may have a 
different mission or may not have the academic expertise to offer a quality 
program on that subject A continuing education program for nurses may 
require an instructor with expertise in physical assessment, but you may not 
have access to faculty members with the right qualifications. 

The converge is also true. Just because another provider offers a program 
does not mean you should not offer that program or one similar to it The 
demand may be strong enough for more offerings. Large numbers of 
classroom teachers may need instruction in the latest approacn to elemen* 
tary science education. You may attract a different clientele than another 
organization would, or you may draw your audience from a different part of 
the country. When considering whether to offer a similar program, ask 
yourself the following questions. 

1. Does our organization have a similar mission? 

2. Do we have faculty members who can address this topic? Are they 
willing to offer the program? 

3. Are there enough potential participants in our market area to make this 
program feasible? Will they recognize that our organization has the 
expertise to offer it? 

4. Are more offerings of this program needed, or is the market saturated? 

5. Do we have ^ sufficient funding base to offer this program? Will 
people enroll if we have to charge a higher fee? 

When studying other providers, however, remember your own ideas. 
What do you think is needed? What programs do you think would be 
effective? Do notgetso distracted by what other organizations are doing that 
you fail to cultivate your own perspective on program planning. 

Professional Development Activities 

How do you get your ideas? Professional development activities are one 
way. Conference sessions that discuss the learning needs of professional 
groups can suggest specific program ideas. Other sessions that address 
issues of importance to that profession can yield insights into how to appeal 
to that group. Conversations with participants, however, often are more 
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helpful than the sessions. These conversations can help you understand what 
will motivate people to attend your programs and what resources they have 
to do so. 

Literature 

Professional literature thatdiscusses learning needs or gives insights into 
the profession is obviously important, but other literature is helpful too. For 
example, educators and others have given us their ideas on future changes 
that wHl influence our society. Harold Hodgkinson (1985), while claiming 
not to 6e a futurist, charts the future implications of demographic changes 
that have already occurred. What do these changes suggest for continuing 
education? 

Other demographic literature is available as well. Thanks to computer 
databases and an increased interest in marketing, periodicals such as 
American Demographics, The Futurist, and TheJohnNaisbitt TrendLetter 
have emerged lhatdiscuss demographic and lifestyle changes. Sourcebooks 
provide household population, income, age, and racial information broken 
down to the individual county level. Some publications profile purchase 
patterns and lifestyles. Others summarize consumer market research stud- 
ies. These publications may not give you specific program ideas, but Jiey 
do offer insights into social changes in our culture. 

Some of us tend not to read. We are active people who like to get the job 
done. We prefer conversation to crawling into a comer with a book. That is 
why we enjoy program planning. We like to be busy and feel uncomfortable 
if we are n OL When colleagues ask how we are doing, we make sure we tell 
them we are busy. We would feel guilty and perhaps judged negatively to 
say otherwise. 

Do not overlook the value of reflection. If possible, jet aside two hours 
each week or a half day every month. Or take a full day every six mohths. 
Find a quiet place out of the office to read and think about what you are doing 
and where you would like to go. 

If we fail to develop the capacity to read and reflect critically, sooner or 
later we will ease right into irrelevance. Nowlen (1980, p. 14) reminds us 
that "the origins of the continuing education programs lie not simply in 
evolving Jfe roles, new government legislation, community problems, the 
increasing employment of women, or additional leisure time bul in the 
minds of adult educators." 




Chapter 3 
Formal Market Research 

In addition to ongoing informal market research activity, formal research 
projects are important. We will discuss methods for collecting data both 
from individuals and from groups. We will describe each methodbriefly , list 
advantages and disadvantages, and mention key points to consider for use 
with conu'nuing education programs. Because this discussion cannot be 
exhaustive, consult the references listed at the end of this publication for 
more information. 

With Individuals 
Surveys 

Conductingasurvey maybcthemostfrequenlapproach to formal market 
research in continuing education. If not the most frequent, it is at least the 
method new program planners choose first. Typically, a sample of potential 
participants are asked what educational needs they have and which continu- 
ing education programs they would attend. Ora selectgroup of professional 
or community leaders is asked what their constituency needs to learn. 

Surveys can be conducted by mail, by telephone, or in person. Our 
discussion will focus on mail surveys, the method used most often in 
continuing education. Later sections will discuss both telephone and face- 
to-face interviews, which have some similarities to surveys. 

Advantages 

1. Well-known method. Most program planners, even if new to their 
position, have some idea of how to conduct a mail survey. Through graduate 
or undergraduate research methods courses, they may havebeen intxuduced 
to survey research. At the very least, they, along with most of the general 
public, have undoubtedly received questionnaires in the mail that asked 
them to evaluate a magazine they read or to answer questions about their 
lifestyle. Because mail surveys are so widely used, most potential respon- 
dents are familiar with llie method. This familiarity may influence them to 
participate in a survey. 

2. Wide distribution. A mail survey can reach everyone with an address, 
courtesy of the postal service. 

3. Low cost. For many market research projects, one professional staff 
member with survey research skills can conduct virtually the entire survey 
from start to finish. Some secretarial help is needed as well. Aside from staff 
time, the only direct costs are for printing and mailing. 
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4. Valid data. When people lake time to complete a questionnaire, they 
typically give good information. Because they can answer in the privacy of 
iheT homes or offices and know their responses can be kept confidential, 
they are less likely to give socially desirable answers than in an interview or 
group discussion. 

Disadvantages 

1. Needfor mailing list. Obtaining a good mailing list can be difficult or 
expensive. Unless a survey will clearly benefit their membership, many 
organizations are reluctant to release their mailing lists. All varieties of 
mailing lists can be rented from list brokers, but renting adds to the cost of 
the project. 

2. Declining response rate. During the past two decades the response 
rate for mail surveys has declined (Frey , 1983, p. 39). Because mail surveys 
have become a popular way to assess educational needs, as well as to study 
various facets of human behavior, people may resist completing yet another 
questionnaire. Some people may wish to protect their privacy. Others may 
have completed so many surveys thai they simply toss the next one they 
A.A.oive. Or they may set it aside for a while, envisioning some leisurely 
Friday aftemoon when they will have lime and inspiration to complete the 
survey. More often than not, thai Friday aftemoon never comes, and the 
questionnaire quietly dies a pigeonhole death. The only respondents eager 
to mail surveys may be people who have time on their hands, have never 
been surveyed before, or have great sympathy for social science u s<^ch. 

3. Lengthy process. A comparatively long time is needed to completf^ 
data collection. Once the questionnaires are in the mail, the respondents 
control how quickly information is obtained. Two months is a typical time 
period to allow for responses to an initial and a follow-up mailing. 

4. Limited data. The type of questions used for a mail survey is limited. 
Questionnaires must be understood easily and not demand too much effort 
from the respondent The type of responses is also limited, frequently 
multiple choice or rank order. Somequestionscan ask forashort answer, but 
generally questions requiring a long answer will discourage respondents 
from answering. Complex or follow-up questions cannot be asked to clarify 
meaning or seek additional information. Emotional intensity and commit- 
ment to participate in a continuing education program are difficult to 
determine from questionnaire responses. 

5. Discrepancy between questionnaire responses and actual behavior. 
Data collected from a mail survey may indicate a need for a particular 
continuing education program. When that program is offered, however, too 
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few people may enroll One reason for the discrepancy is that respondents 
tend to express interest in programs with which they are familiar. Yet that 
expression of interest does not indicate they will take the time and find the 
money to enroll (Nowlen, 1980, p. 32). Another reason for the discrepancy 
is that respondents are offered too many choices. Often the questionnaire 
presents a long list of pDssible perceived needs and an equally long list of 
potential continuing education programs. The respondents may check all 
the needs they feel and all the programs for which tliey have any interest at 
all. As a result, the mail survey does not provide a significant opportunity 
to evaluate motivation to participate in particular programs. 

Key Points 

1. Pilot test the questionnaire. You may be tempted, especially if you 
have experience with survey research or are pressed for lime, to develop a 
questionnaire too quickly. Do not try to draw up a questionnaire in the 
morning and pop it into the afternoon mail. Share sample questions with 
colleagues to get their reactions, and then develop a draft questionnaire. 
Select a small sample of up to twenty people similar to the population you 
have chosen to study, and administer the quesuonnaire to this group. Ask 
them to include both their answers to the questions and tJicir reasoning in 
answering the questions. That way you will know if your quesuons are 
communicating what you intend. 

2. Ask questions in the most appropriate order. Give the respondent a 
chance to w arm up with easy, nonthreateningquestions. PLcc rhore difficult 
or threatening questions near the middle of the quesuonnaire after respon- 
dents have hit their stride. Leave questions about age, income, education, or 
other personal information for the end. 

3. Keep the questionnaire as short as possible. We all have a tendency 
to makea questionnaire too long. As long as I have your attention, we reason, 
I might as well include a few more quesuons. A lengthy questionnaire can 
conuibute to a low response rate. Avoid the other exu-eme, however, and do 
not make the questionnaire too superficial. Ask the questions that you need 
to ask. 

4. Design the questionnaire to look professional. A questionnaire can be 
atuactive without costing a lot more to produce. Printing it .n booklet form, 
using tan, gray, or another neuual color, can usually be dciie inexpensively 
by an in-house copy center. 

5. Code the questionnaires for tracking responses. a numbering 
s>stcm that enables you to determine who has and has not responded. Place 
the respondent number on the questionnaire in an inconspicuous place. 
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6. Include a cover teller. Stale the purpose of ihe survey, what it will 
accomplish, what organization is sponsoring it, and how the respondents 
have been selected. Explain the purpose of the code number, and assure 
respondents you will treat their responses confidentially. Give a date to 
return the survey, typically two or three weeks. 

7. Send a follow-up mailing. One week after the return date, send a 
reminder letter and another questionnaire to those who have not responded. 
Ask them to retun^ it as soon as possible, but indicate another due dale. 
Another way to follow up is to send a reminder letter or postcard on the 
return date to all respondents, asking them to complete the survey if they 
have not already done so. 

Delphi Technique 

The Delphi technique was developed in the 1950s by the Rand Corpora- 
lion as a way to forecast national defense needs. It takes its name from the 
Delphic Oracle of ancient Greece, whose function was to help predict the 
future. Today the Delphi is a good market research technique, both for 
projecting future needs and for confronting current problems. 

The Delphi is a series of mail surveys designed to develop consensus 
about a particular problem or issue. The first questionnaire typically is 
limited to one or two open-ended questions. The responses are tabuidted, 
sorted into categories, end compiled into a list of problems or issues. This 
list is incorporated mto a second questionnaire, which asks the respondents 
to rate the importance of each problem or issue and to offer any reactions to 
the list. For example, respondents may argue for otlier problem s or issues too 
important to omit. A third questionnaire may be developed to ask the 
respondents to rank order the top five or ten vote-getters. It may also include 
a summary of comments included on the second questionnaire and ask for 
additional reactions. The end result, then, is consensus about what the 
r?spondents believe to be the most important problems or issues. 

In continuing education the Delphi might be used to determine what 
skills a profession will need in the future or what demands for services an 
institution will have to accommodate. The process also might be used to 
predict how a community will change or what needs that community will 
have for educational programs. 

Advantages 

1. High participanl motivalion. Because Delphi participants are care- 
fully sclccte<l, they are usually more highly motivated to complete the 
questionnaires than are regular mail survey participants. They typically will 
take the time to offer thoughtful responses. 
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2. Sense of closure. Unlike respondents to a mail survey, Delphi partici- 
pants know how others have responded to the questions. This feedback and 
the subsequent questionnaires help them feel that their ideas arc leading to 
some resolution. Feedback also helps them learn more about the particular 
issues 

3. Widely scattered participants. The Delphi is a manageable process 
that can include geographically dispersed respondents. Organizing a meet- 
ing for them to develop consensus on a particular problem or issue would be 
much more difficult and expensive. 

Disadvantages 

1. Limited participant interaction. Anticipating future needs is often 
difficult. Having a chance to talk about the subject with other people can 
help project beyond the demands of the present. Delphi participants, 
however, must respond to the first questionnaire without the benefit of 
interaction with other people. Although some feedback from other partici* 
pants may be included on subsequent questionnaires, they cannot discuss 
that information among themselves. Because of this limited interaction, 
responses to a Delphi process may be less thorough and creative than 
responses that the same people would generate if they could meet face-to- 
face to discuss the issues. 

2. Respondent dropout. Although participant motivation may be high, 
inevitably the sample gets progressively smaller. Each time a questionnaire 
is distributed, some people will not return theirs. 

3. Time-consuming process. The Delphi takes almosias long tocondact 
as three separate mail surveys. Some time can be saved by giving the 
participants less time to return their questionnaires than for a one-time mail 
survey. Because these individuals have agreed to participate in the process, 
they can be expected to return their surveys sooner. 

Key Points 

In addition to the key points for conducting a mail survey, the following 
points are important for the Delphi technique. 

1. Select knowledgeable, future-oriented participants. Delplii partici- 
pants must be well informed about the particular profession, organization, 
or community under consideration.The> also need the ability to sec beyond 
the present situation into the future. Not everyone who is well informed 
about present issues is able to develop this future orientation. 

2. Select the appropriate sample size. The Delphi usually calls for a 
more selective sample than a general mail survey. For some projects with 
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high participant motivation, only 10 to 15 carefully chosen individuals arc 
necessary (G ilmorc, Campbcl I, and B ccker, 1 989, p. 45). For other projects, 
a larger sample is preferable to protect against dropouts. To estimate the 
appropriate sample size, assume that for each round as many as 25 percent 
will not return the questionnaire. 

3. Allow less time for return of questionnaires. Instead of allowing three 
or four weeks, ask participants to respond in two or three weeks. If a follow- 
up mailing is done, give them only one or two more weeks to respond. 

4. Decide how to handle dropouts. People who fail to return the first or 
second questionnaires may be eliminated from the next rounds. Or they may 
be given another chance to respond to what the oihei participants generated. 
If nonrcspondents are eliminated from future rounds, consider the effect on 
sample size. With a 25 percent dropout rate each round, only 48 of 64 
individuals may return the first questionnaire. For the second round, only 36 
of the remaining 48 participants may return the next questionnaire. For a 
third round, only 27 of the remaining 36 may return the final questionnaire. 
The result would be a final response rate of only 42 percent. Because of this 
significant attrition, including all original participants in each round is 
desirable. Do so, however, only if inclusion of non-respondents will not 
skew the results. 

Face-to-Face Interviews 

Interviewing can be an exf,it ing way to conduct maitet research because 
it provides the chance lo talk directly to respondent^). If you like talking with 
people one-on-onc in a face-to fate netting, interviews can be a fascinating 
opportunity to gain Insights into people's needs and motivations. 

Interviews can be formal or informal. Informal interviews are similar to 
ordinary conversation. They are an example of informal market research 
and will not be considered in this section. 

Formal interviews can be highly scheduled, moderately scheduled, or 
nonscheduled. A highly scheduled interview follows a very specific set of 
directions. Intervicwersmustaskal! the questions on an interview schedule, 
using the exact words and following the same order. They are told what 
follow-up questions to«^k if certain responses arc received. Nonsc! cduled 
interviews allow the interviewer much more freedom. Interviewers are told 
only the goals of the interviews. They must then decide what questions lo 
ask and in what order to obtain the desired information. 

Moderately scheduled interviews fall between these two extremes. 
Interviewers typically receive a list of questions to ask, but they are free to 
vary the way ihe Interviews are conducted. They can, for example, ask the 
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questions inadiffcrcntordcrjollow up highly productive rcspoiiscsalgrcat 
length, or skip questions that clearly do not apply or have already been 
covered. 

Advantages 

1. Good response rate. Many people who will not return a mail survey 
will agree to be interviewed. They may be very busy, bat somehow they can 
squeeze another appointment into their schedule. People like to talk aDout 
themselves and feel important when singled out for their perspectives. 

2. Immediate information, UnliI.e survcy.s, which icquirc a lengthy wait 
for mail returns, interviev;s provide information quickly. As soon as the 
interview begins, the interviewer can develop impressions of educational 
needs and motivation to participate in programs. 

3. Wide range of information. An interview provides the opportunity to 
ask a greater variety of questions than in a mail survey. More complex 
questions can be included, because the interviewer can explain their 
meaning if the respond entdoes not understand. More open-ended questions 
can be used, because respondents typically prefer to talk than to write. 
Interviews also allow follow-up questions to clarify the meaning of re- 
sponses or seek additional data. If the interview is not highly scheduled, 
these follow-up quesUons can lead to discovering unanticipated informa- 
tion. 

4. Insights into motivation. Compared with a mail survey, an interview 
offers significantly more opportunity to sense emotion. Responses to a mail 
survey must be taken at face value, but an interviev, can reveal how strongly 
people feel abou t their smtemenis. As a result, the interview can suggest how 
motivated respondents arc tu participate in a continuing education program. 
Even if respondents arc not asked directly about their motivation, the 
interviewer can sense genuine interest or simply mechanical responses to 
the questions. 

Disadvantages 

1. Need for trained interviewers. The extent of interviewer training 
depends on how scheduled the interviews arc. For highly scheduled inter- 
vic-wa, relatively little training may be necessary. Because interviewers 
recei^'c very specific instructions on the interview schedule, they may need 
only a basic orientation to the project, instructions on recording data, and a 
chance to conduct a few practice interviews. Nonscheduled interviews, at 
the other extreme, place significant reponsibllity on the interviewers and 
requirc them to have some knowledge of the content of the interview. 
Interviews with physicians, for example, would require interviewers who 
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have a working knowledge of continuing education and some insight into 
the world of medical practice. 

2. Difficult da analysis. Highly scheduled interviews yield data simi- 
lar to a mail survey. Except for a few more open-ended questions, most of 
the data arc forced-choice responses, which can be analyzed quantitatively. 
For moderately scheduled and nonscheduled interviews, however, most, if 
not all, of the data require a more difficult qualitative analysis. 

3. Costly process. Interviews result in more staff time and direct ex- 
penses than do mail surveys. Interviewer training and dataanalysis are two 
time-consuming activities. Interviewers may have to be paid for time spent 
conducting the interviews and traveling to and from the interview site. 
Travel expenses are typically greater than postage for a survey. 

Key Points 

Pilot testing questions, asking questions in the right order, and keeping 
the list of questions as short as possible apply equally to interviews and 
surveys. The following key points apply especially to interviews: 

1. Introduce the project in advance of the interviews. Call respondents 
to ask Ihem to participate in the interviews. State the purpose, goal, and 
sponsorship of the project. Explain how the respondents were selected, and 
assure them that -esponses will be kept confidential. Arrange for an 
interview at their office or other convenient location, and state how long the 
interview will take. For especially busy professional people, you may want 
to send a brief letter explaining the project a few days before the initial 
telephone call. 

2. Select more respondents than are needed for the interviews. Not 
everyone will agree to participate. Some people will be too busy, others will 
be unavailable when you need them, and others will simply decline. Still 
others will be difficult to contact. Decide how many attempts to make to 
reach them before substituting other respondents. 

3. Choose interviewers who enjoy one-on-one interaction. Interviewers 
need to put respondents at ease and encourage them to talk freely. They 
should be good listeners who know when to interjectsupportive comments 
and follow up questions. They also need to observe and interpret nonverbal 
cues. People who are at ease with a one-on-onc, face-to-face situation are 
best able to satisfy these demands. 

4. Record data in the least cumpliMcd way. In most cases, tape record- 
ing interviews is not advisable. Market research projects do not require 
analysis of every word uttered. Tape recording is also expensive. Transcrib- 
ing a one-hour interview takes from two to six hours (Gorden, 1987, p. 263). 
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Instead, interviewers can lake extensive notesduring the interviews. Or ihey 
can limit themselves to key words and phrases to help vmte a detailed 
summary as soon as possible after the interview. 

Telephone Interviews 

An alternative to face-to-face interviews is to use the telephone. Tele- 
phone interviews are especially appropriate if you or your respondents are 
pressed for Ume. They work besias highly scheduled interviews. Compared 
with face-to-face interviews, telephone interviews have several advantages 
and disadvantages. 

Advantages 

1 . Belter response rate. Some people who are too busy for a face-to-face 
interview or are uncomfortable with it will take a few minutes for a 
telephone interview. 

2. Moreimmediateinforrnation.LikQ face-to-face interviews, telephone 
interviews provide information quickly. They can offer even more immedi- 
ate feedback if the interview can be conducted with just one call. 

3. Less interviewer training. Because telephone interviews typically are 
highly scheduled, the interviewers need less training. Some training is still 
necessary and some knowledge of the project is helpful. But the interview 
schedule gives them specific questions to ask and instructions to follow. K 
appropriate, staff members or volunteers in the organization can help with 
interviewing, which contributes to their own professional development 

4. Less difficult data analysis. Because telephone interviews typically 
use a highly scheduled approach, more quantitative data are obtained than 
in face-to-face interviews. Qualitative data from telephone interviews are 
usually short answers which can be analyzed quickly. 

5. Less costly process. Telephone interviews normally require less time. 
They do not lake as long to conduct and do not require travel. Face-to-face 
interviews may last as long as an hour, but phone interviews for market 
research should be limited to fifteen or twenty minutes. Five or ten minutes 
may often be enough. For some projects, long distance telehone calls are 
necessary, but they cost significantly less than travel to faraway places. 
Since data analysis is simpler, it will require less staff time. 

Disadvantages 

1. More limited information. Obtaining thorough responses to open- 
ended questions is more difficult on the telephone. Respondents lack the 
verbal and nonverbal cues that face-to-face interviewers provide to encour- 
age thoughtful and detailed answers. As a result, typically a response is 
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shorter than it would be if respondents were asked the same question in 
person. The type of questions appropriate for a telephone interview is also 
more lim ited. Respondents find complex questions more difficult to keep in 
mindonihephonelhan they would face-to-face. Fewer follow-up quesuons 
are possible as well. 

2. Less insight into motivation.TQlQi^honQ interviewers have somesense 
of how strongly respondents feel about their ano, ers and how motivated 
they are to partis^ipate in a continuing education program. They cannotsense 
the full range of emotion present in a facc-lo-face interview, however, 
because they have to depend on voice alone. 

Key Points 

The key points for telephone interviewing are similar to face-to-face 
interviewing, with two exceptions. 

1. Select interviewers who are at ease on the telephone. Telephone 
interviewers must establish rapport quickly mu respondents, using only 
their voice. They should, therefore, have pleasant voices and enjoy taiki.ig 
on the telephone with people they do not know. 

2. Rely on note-taking to record data. Tape-recording is even more 
inappropriate than for face-to- face interviews. Since telephone interviews 
follow a structured schedule of questions, most answers can be recorded as 
the interview proceeds. Open-ended questions may require additional 
wriunp and editing immediately after the interv iew. It is belter to take a few 
notes for those questions during the interv iew and add to them later than to 
have long pauses during the interview. 

With Groups 

Data can also be collected from an entire group. Having a group of 
respondents together at the same lime can sav e consi dcrable cost, compared 
wiih contacting ever>onc individually. The interaction among individuals 
in a group can generate idcai> that might not emerge on an individual basis. 

Gatl.s:ring an entire group togellier can be difficult, however, especially 
professional people. Contacting everyone individually can be simpler than 
organizing a meeting. A group setting can also result in fewer responses. 
Time does not always permit group members to say what they would say if 
contacted individually. Some people may feci inhibited in a group if they 
have conflicts with other group members or perceive significant status 
differences. 

We will discuss two group methods, nominal groups and focus groups, 
which are especially suited for formal market research. We will mention 
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only ihc advantages and disadvantages of using each method in continuing 
education, but the points we have just made about working with any group 
also apply. If you are unfamiliar with these methods, be sure to consult 
references at the end of this publication. Our discussion is loo brief to 
provide the detailed information necessary to use the methods effectively. 

Nominal Group 

A highly suiiciurcd approach to market research, the nominal group 
technique involves limited interaction among group members. Thus a 
nominal group is a group in name only. Originally designed for use in 
business organizations, the entire process includes five stages. Each stage 
involves "increasingly higher levels of authority within the organization. 
Lower level workers identify theproblems, but upper-managementperson- 
nel decide what programs to conduct and what resources to allocate" 
(Campbell, 1980, p. 17). 

In continuing education, nominal groups are often li.r.ued to the initial 
problem identification stage. One or more groups of five to seven partici- 
pants respond to a single question. For example, they might be asked what 
economic challenges their community faces or what jproblems they are 
experiencing in their professional practice. WiJioiit coniulting other g;oup 
members, participants write responses to ihe qu^isiion on a note card. In 
round-robin fashion, they then shaie their rcspoiwCi, one answer at a time. 
Each different response is listed on a flipchart. 

After all responses have been shared and recorded, participants vote for 
the mosiimponant responses. First, they select aslated number of items they 
think are most important and rank them in priority order. These votes are 
tab ulated and may be discussed further, followed by a final vote. Each group 
ranking is then tabulated to form a composite ranking (GiLnnore et al.,1989, 
p. 65). The composite ranking becomes the basis for program planning. 

Advantages 

1. Equal opporiunity for participation. Unlike other group methods, a 
few talkative members cannot dominate the group. Everyone has the same 
opportunity to offer responses to the group, and the group must consider 
every response. Gilmcre et al. (1989, p. 62) claim that "because of the 
disciplined process, minority opinions and conflicting ideas arc tolerated." 

2. Thoughtful responses. Participants must take time to write their 
responses before sharing them with the group. Extroverts may therefore 
offer more insightful responses than ihcy m ight in a brainstorm ing group or 
open discusssion session. Introverts cannot sit back and let the more 
talkative people carry the load. Seeing others writing encourages them to 
consider the question as well. 
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Disadvantages 

1. Limited data. The nominal group can respond to only one question. 
Other group methods can consider as many questions as time allows. 
Because interaction among group members is restricted, responses are 
limited to those that members can generate individually. Ideas that might 
result from typical group interaction are lost. 

2. Lengthy process. If the entire nominal group process is used, five 
different stages with five different meetings of five different groups of 
people are required. 

Key Points 

1. Develop a good question. The question "should be clear and simply 
stated" (Gilmore et al., 1989, p. 63). Since only one question is used, it has 
to be good. Generate several alternative questions and pilot test them before 
selecting one for the nominal group. For example: What is the major 
problem you are facing in your professional life at this time? On What is the 
major problem in your community at this time? 

2. Choose knowledgeable participants. Nominal group members need a 
thorough understanding of the relevant issues, because they must first 
generate their own written responses without discussion and then prioritize 
the group's responses. 

3. Choose disciplined participants. Group members need the ahiUty to 
work well with the structure that the nominal group imposes. Discussion is 
more controlled than in most other groups. Participants offer their ideas in 
specific ways at specific times. 

Focus Group 

The focus group has its roots in psychiatry and was developed in the 
1950s for marketresearch. Today it is widely used in business for marketing 
purposes and has a growing use in continuing education. A focus group 
typically includes six to twelve people, reasonably similar in age, occupa- 
tion, ethnic background, or other demographic characteristics. 

Although comparable in size to a nominal group, a focus group is 
conducted quite differently. It is considerably less structured, resembling a 
moderately scheduled group interview. The group considers a specific 
topic, but docs not limit itself to just one question. The moderator has to 
"focus" the discussion on the topic, "skillfully guiding the discussion in a 
way that stimulates interaction and encourages the sharing of feelings, 
attitudes, and ideas from all group members" (Gilmore etal., 1989, p. 69). 
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Advantages 

1. Wide range of information, A focus group has some of the same 
advantages of a moderately scheduled interview. Open-ended quesuons are 
asked to encourage expression of ideas and emotion. The moderator has a 
list of questions to ask but, skillfully using follow-up questions, can pursue 
in greater detail partic ular issues that concern the group. Unlike m embers in 
ijie nominal group, those in the focus group may discuss and evaluate each 
other's ideas, which can lead to expression of still other ideas. 

2. Relatively short meeting. Although preparation and data analysis can 
be umcconsuming, the focus group meeUng itself can be lim ited to one hour. 
A short meeting can encourage busy people to participate. 

Disadvantages 

1. Need for trained moderators. Moderating a focus group is more 
difficult than facilitating a nominal group. The moderator must know how 
to keep the discussion on track and when to follow up responses with 
additional questions. The moderator also must have good group dynamics 
skills and know how to encourage everyone to participate. 

2. Unequal participation. Quick-thinking and talkative people may 
dominate the discussion. As a result, some people will not nave the same 
opportunity to present their ideas and have them evaluated fairly. 

3. Difficult data analysis. Just as moderating a focus group is more 
difficult than conducting a nominal group, so isanalyzing the resulting data.. 
As in a moderately scheduled or nonschcduled interview, ih^^ open-ended 
questions result in extensive qualitativ e data. Because these data come from 
an entire group, analyzing them can be especially difficult. 

Key Points 

1 . Select a skilled moderator. For an effective focus group, the modera- 
tor **must have good interpersonal communication skills and be able to 
quickly establish rapport and gain the confidence of the participants" 
(Gilmore et al., 1 989, p. 70). If more than one focus group will beconducted, 
try to use the same moderator for each group. 

2. Strive for homogeneous groups. Focus group members need some 
knowledge of the topics to be discussed and ideally should not know each 
other. To encourage the best discussion, recruit participants similar in age, 
gender, ethnic background, edu^^^lion, income, or some other combination 
of deriographic variables. Ideally the moderator should fit the group's 
characteristics as well. If group niembers are too dis:>imilar, their percep- 
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lions of the issues addressed will differ significantly from each other 
(Gilmore et al., 1989, p. 71). An ongoing group has more time to work 
through these different perceptions, but a focus group meets only once for 
ashort time.Itshould therefore bestructured to minimizedifferences among 
group members. Each focus group can be very different from the other 
groups, but within the group the members should be as similar as possible. 

3. Invite more participants tha\ are necessary. Typically everyone will 
not come. If you invite twelve ca efuUy chosen people to each group, half 
that number is still the minimum ize necessary for an effec live focus group. 
If they all come, the maximum group size will not be exceeded. 

4. Consider incentives for participation. People invited to participate in 
a focus group may not identify with the purpose of the group as strongly as 
those selected for a nominal group. Some incentive may be necessary to 
motivate them to participate. Business organizations using focus groups for 
marketing purposes typically pay participants from S15 to $100. If partici- 
pants are paid, the total could be as low as $100 or as high as $5,000, 
depending on the rate and the number of groups used. Monetary rewards 
may not be possible, but including a meal or some other gratuity can 
encourage participation. 

5. Consider tape recording the discussion. Because focus group meet- 
ings typically are free-flowing and only moderately structured, an accurate 
record of the discussion is necessary for later analysis. Videotaping is 
desirable but may be too expensive and not readily available. Audiotaping 
can be adequate, especially if the moderator makes note of significant 
nonverbal behavior during the discussion (Gilmore etal, 1989, p. 73). 

6. Use an observer. The moderator must concentrate on guiding the 
discussion and cannot fully attend to what the group is communicating. 
Studios with one-way observaaon capacity are sometimes used, but they 
will not be feasible for most continuing education market ^^esearch. An 
observer in the room seated away from it can scrutinize the group unobtru- 
sively. This person can record nonverbal behavior and reflect on the 
discussion. The observer can also critique the moderator's leadersh p and 
make suggestions for later focus group meetings. 

7. Continue focus groups until new ideas become infrequent. If partici- 
pants in each group have similar demographic characteristics, new ideas 
typically cease after three or four focus groups. If each group is quite 
different, however, more focus groups may be necessary. 



ERLC 



33 



Fornial Market Research 29 



These twomelhCKis are not ihe only ways to conductmarket research with 
groups. They are merely two very dif feren t methods, chosen because of ihcu: 
growing use in continuing education. Other group methods include plan- 
ning groups, brainstorming groups, and group observation. 
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Chapter 4 
Importance of the Program Planner 

This discussion of formal strategies is not intended to suggest that market 
research is simply a straightforward, empirical process. Determining pro- 
gram directions does not happen automatically from a formal research 
project. Although formal market research can play an importantrole, "most 
administrators and program planners rely heavily upon their experience and 
intuitive judgment in generating program ideas" (Nowlen, 1980, p. 17). 

Program planners are critical to market research, but their importance is 
not always recognized. Instead, planners sometimes feel they should keep 
their distance and take a mechanical approach to market research. Jerold 
Apps (1985, p. 174) claims that "making decisions about what is ideal or 
what should be is something many programmers shun. They proclaim that 
their role is one of assessing needs, not determining needs." 

Program planners, though, cannot remain aloof from the process. Apps 
further argues that needs assessment is not a simple value-free, empirical 
procedure. Data can be collected empirically, butplanners musldecide what 
to do with these data. They must judge what the data mean for continuing 
education programs. Apps claims that a programmer must "establish 
priorities concerning which interest areas to program for and thus is making 
a value decision. ... It is impossible for a programmer in continuing 
education to remain neutral" (Apps, 1985, pp. 175-176). 

Knowing a variety of formal and informal marketrcsearch methods is not 
sufficient You must decide how to use these methods to benefit your 
programming. Your own philosophy of education will influence your 
choice of methods and how you use them, as will personality and past 
experiences with research methods. Your working environment is also a 
critical factor because time, money, and personnel available will limit the 
scope of market research activities. We must now consider how these three 
factors— your philosophy of education, personal preferences, and working 
environment— all influence your marketrcsearch. 

Philosophy of Education 

Apps (1973) stresses the importance of a working philosophy of educa« 
tion. He claims that what we believe about adults as learners, the purpose of 
education, the nature of the curriculum, and the learning process all affect 
our approach to continuing education. He urges us to identify our beliefs 
about these components of education, examine them for any inadequacies 
or contradictions, and change our beliefs when necessary. 
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Wc may think of a working philosophy of education, if we ihink of it at 
all, as applying to the educational program itself. Yet it is also an important 
foundation for market research. What we believe about the adult learners we 
serve, the goals of our continuing education programs, the content of these 
programs, the role of our faculty, and our role as program planners should 
be consistent with the approach we lake with market researcii. 

Two questions illustrate how ourphilosophy of education influences our 
market research. First, what is the purpose of our continuing education 
programs? Second, who should determine the goals and curricul um of these 
programs? Learners? Professional experts? Supervisors in the workplace? 
Community leaders? Or you, the program planner? 

Purpose of the Program 

Considering the purpose of oui continuing education programs leads tc 
several other questions. Do you believe your programs should help people 
adapt to changes occurring in our society? Or should they challenge people 
tj work for change in their piofessions and in their communities? Are 
changes occurring that improve our communities, or do they result in 
harmful consequences? Do you believe your programs should keep up with 
what other providers offer? 0. do youwanttodevelopadistinctivenichefor 
your organization's offerings? 

What you believe about the purpose of your programs relates to who 
should determine goals and curriculum. If you want to help people adapt to 
change, you need to determine what changes have occurred and what 
concerns people have as a result. You can gather some of this information 
informally through readii.g and talking with colleagues. It is critical, 
however, to ask people directly how Uieu^ lives have changed and what 
difficulties they are experierxing. Surveys and interviews are excellent 
methods for obtaining this information from potential participants in your 
programs. For a program that i ims to help people adapt to change, then, the 
learners play an important pari in determining goals and curriculum. 

On the other hand, if you want your programs to bring a more prophetic 
message, challenging the easily adopted, widely held assumptions underly- 
ing contemporary cultural trends, the learner may not be as important for 
marketresearch.Profcssionaland community leadersca.i identify problems 
in our society and offer their ideas for change. Several methods are useful 
for gathering this information, including informal conversation, surveys, 
and nominal groups. Your perspective is especially important a?, well. Be 
sure to cultivate your own ideas through reading, informal conversations 
with colleagues, and reflection. 
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Program catalogs and brochures from other continuing education pro- 
viders are always a useful source of ideas. If yon wantyourprograms to have 
their own identity, however, you do not wani tO offer every program other 
organizations do. Your own ideas, and those of colleagues in your organi- 
zation, are critical for developing a special niche. The more creative these 
ideas arc, the more distinctive your programs will be. 

Deciding Who Determines Goals 

The way you do market research will be influenced by who you believe 
should determine program goals and directions. If you believe that learners 
shoulddetermine their own needs, be sure to ask potential participants what 
their needs are. Surveys, interviews, and focus groups, for example, are 
good methods for obtaining felt needs directly from learners. If you believe 
that professional and community leaders are a better source of information, 
the Delphi and nominal gix)up techniques are especially appropriate. If you 
bclieveihat the program planner should havea major influence on program 
goals and cumculum, be sure to cultivate your own ideas through informal 
market research. 

We do not mean to imply, however, that you should never do informal 
market research if you believe adults should detemiine their own learning 
nc^ds. Nor do we mean that you should never use surveys or nominal groups 
if you believe the program planner should have a strong influence on 
program goals and cun-iculum. What we do mean is that some market 
research approaches are more consistent than others with your own personal 
philosophy of education. 

We especially want to wam against market research approaches that are 
inconsistent with your philosophy of education. If you believe, for example, 
that learners should determine their own learning needs, do not limit your 
research to personal reading and reflection. By themselves those activities 
would be inconsistent with your philosophy. Be sure to employ research 
methods that allow learners to express their needs to you. Similarly, if you 
believe that the program planner should propose program ideas, do not 
spend all your Ume collecting data from potential participants. Be sure your 
research activities help you formulate good ideas. 

Personal Preferences 

In addition to your philosophy of education, personal preferences influ- 
ence how you do market research. Your personality and working style 
enable you to enjoy some activities, and succeed with them, more than 
others. Past experiences with certain research methods have helped you 
develop skills for those methods. 
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Because of these personal preferences, planners will gravitate to some 
research methods more than to <^ihers. Do you like working with groups? If 
so, you may think first about forming a planning committee before doing 
any market research. Or you might want to organize a focus group or 
nominal group exercise for community leaders. T>o you enjoy talking with 
people one-on-one? If so, you may look forward to a scries of face«to«face 
interviews with potential participants. If these interviews make you uncom- 
fortable, you may prefer the telephone. 

Or do you prefer working alone? Have you had your fill of meetings? Do 
you like developing your own ideas better than collaborating with others? 
Your favorite form of market research may be crawling into a comer of the 
library with the latest professional journals and popular periodicals. Do you 
like daU\ analysis and writing? If so, you may think first of a mail survey. 

The balance achieved between reading, th inking, and talking wi th people 
will depend on your personality. Many people attend conferences to meet 
new people, talk with them about their programs, and come away with new 
ideas. Some of us look forward to conferences as a time to reflect Great 
idc as can emerge in the middle of a session, whether boring or exciting. 
These ideas have very little to do with talking to new people. They have a 
lot to do with the way we think and get new ideas. 

Using your preferred research methods is often appropriate. Understand 
the approaches you prefer and accept your tendency to gravitate to them. If 
you like a particular approach and are comfortable with it, you will work 
hard at it and do it well. If you have had experience with a certain method, 
you will have developed skills to use that method well. Use these methods 
especially when time and money are at a premium. 

Using only your preferred approaches, however, can be too limiting. If . 
you always conduct a mail survey with potential participants, you will have 
only their ideas on what is needed. You will not have the perspective, for 
example, of leaders in their community. You will also continually gather 
information that has inherent mail survey biases. If you always limit market 
research to library activity, you will not know ifanyone besides yourself has 
any interest in coming to your programs. When the time is right, try to 
broaden your repertoire of market research skills. Move out of your comfort 
zone, and use the opportunity for professional growth. 

Working Environment 

Your v/orking environment also governs your approach to market 
research. Given yourphilosophy of education and personal preferences, you 
know the methods you would like to use. You need the opportunity, 
however, to use them appropriately in your own organization. 
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Wc sometimes think of market research as a major project, involving 
several staff members and thousands of dollars. Although a few projects are 
that large, most are on a much smaller scale. Nowlen (1980, p, 25) cites one 
study which found that, for universities at least, "most of the research 
con- .acted to determine program ideas is done in house by personnel for 
whom conducting research represents only one of their responsibilities." 
Maiicei research is probably only one of your responsibilities as well. The 
challenge is to fit market research into all the other demands of your job. 

Personnel and Financial Resources 

One challenge is to find the necessary resources. If your organization 
docs not specialize in research, an expert staff will not be at your command 
to help with research design, data collection, and analysis. You may also 
have limited financial resources with which to pay for a research project. 
Time is another scarce resource. You have many other programming 
responsibilities, with their own deadlines and financial obligations. You 
cannot devote as much time to market research as you would like. The time 
you do have may not be in extended blocks. 

Given these resource limitations, infonnal maricct research is very 
important. Even with limited time or money, you can always talk with 
colleagues about program ideas, review promotional materials from other 
providers, and read professional and popular literature. 

Organizational Culture 

Another challenge in your working environment is the culture of the 
organization. Your organization has a history of doing market research in 
certain ways that you cannot ignore. Your colleagues may place high 
priority on working with advisory committees to plan progams. Standing 
committees may always plan the annual conferences. To avoid these groups 
while you dream up new ideas in the library would obviously get you in 
trouble. Your colleagues may have strong allegiance to survey research or 
othcrformal methods. They may distrust more informal approaches. Or the 
organization may not have the financial resources to support formal proj- 
ects, encouraging ongv..*»g informal activity instead. All of these factors in 
the working environment will influence thcchoicesyou make about market 
research. 
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Chapter 5 ■ - .. - 
Executing Market Research 

Equipped wilh a repertoire of formal and informal market research 
methods and recognizing the influence of your philosophy of education, 
personal preferences, and working environment, you arc finally ready to 
take the plunge. The ten steps that follow will help you plan and then conduct 
your market research activities. Thinking through these steps will help 
determine what methods to use for market research, how often to do it, how 
cxtensivelyyou should getinvolved,and what roleotherpcopleshouldplay. 

Step 1. Decide What to Research. 

This step may seem lOO obvious :o mention, but you do not have time to 
do all the market research you would like to do or feel you need to do. 
Determine your highest priority for research activity, considering new 
programs you want to begin soon and ongoing programs you would like to 
expand or modify in some way. 

Step 2. Consider the Resources Available. 

For most market research activity, tlie necessary resources include 
people, money, and time. What technical expertise is available from the staff 
in your organization to help with research design, insuument development, 
data analysis, repon writing, and editing? Can some staff members help 
collect and summarize data? Will planning committee members or other 
volunteers help with some aspects of ilie project 

How much money can be spent on this activity? Can you afford travel 
expenses to collect data? If technical expertise is not available in the 
organization, can you hire consultants or free-lancers to help? 

How much time do you have to accomplish this research? Do you have 
to decide next month whether to scrap an ongoing program? Do you have 
a year todc velop a new program thrust? Can you give a research projat your 
undivided attention for the next three months? Or do you have to fit it into 
all your regular programming responsibilities? 

Step 3. Determine the Best ^ime to Do Research. 

When is the best lime for you to conduct market research? When you are 
beginning to plan and budget for next year's prograin schedule? At whai 
time of year can you give your best effort to market research? During the 
winter months when you are not as busy running programs? If you need to 
collect data, when will the respondents be most accessible? When can you 
count on the most staff or volunteer help? 
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Step 4. Choose a Market Research Approach. 

On the basis of your answers to the above questions, decide how 
extensive your effort will be. If you have excellent staff support, a healthy 
budget, and an adequate block of time, you may want to plan an extensive, 
formal research project If you have to do most of the work yourself with a 
limited budget within a short time frame,a quick survey or a few interviews 
may be best. 

Select the method or methods to be used. Consider the advantages and 
disadvantages of each method described earlier. If you anticipate formal 
data collecUon activity, keep in mind the methods that work besi ^jr your 
respondents. Select methods that are consistent with your philosophy of 
education and fit your working environment. If time permits, consider a 
method that requires you to develop new skills. Learning a new method is 
one way to use market research for professional growth and renewal. 

Step 5. Recruit People to Help. 

Research should notbedone in isolation. Although reading, writing,and 
data analysis are all solitary activities critical to the success of any research 
project, other people are important as well. You need them to help formulate 
your plans, challenge your ideas, reflect on the data collected, speculate on 
its implications for programming, critique drafts of reports, and generally 
support your efforts. Stimulation from this interaction will help you work 
through difficult phases of the project and improve the quality of research. 
Involving other people in market research can also help you gain renewed 
perspective on your programming. 

Before plunging too far into research activities, then, find other people 
to work with you. Use an ongoing planning committee or form an advisory 
group just for this project Even if you plan low-key, ongoing, informal 
market research activity, ask at least one other trusted colleague to consult 
with you from time to lime. 

Once somekind of supportgroup is in place, determine die personnel you 
need. You need not be an expert in all phases of market research to do quality 
work. Secure ilic services of staff members within your organization who 
can offer necessary technical expertise, help collect and analyze data, and 
provide clerical and secretarial -support. You may have access to an applied 
research unit that can handle data collection and analysis. Ii ,^ff ...embers 
arenoiavailable intemally,considcryourneedforextemalconsullants,data 
collection assistance, and free-lance wrilcrsand editors. Or recruit volunteer 
help, especially for data collection. 
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Step 6* Develop a Preliminary Plan for the Research. 

The timehascome to plan theprojectJf youplan formal market research, 
select the population to study. Determine the sample size, and choose the 
sampling procedures. Do you want to know what will draw the general 
public to your programs? Or do you want community leaders to suggest 
programs they believe are needed? Do you plan to sample several hundred 
potential participants in your programs? Or will you limit the sample to fifty 
respondents? How will you obtain their names? From the telephone book? 
From membership lists of professional organizations? Or do you plan to 
randomly approach people at street comers and shopping malls? 

Decide how to collect data and what assistance will be needed. Do you 
need to develop aquestionnaire or interview schedule? Do you have the time 
and expertise to do so, or do you need technical advice jfrom other people? 
Will you interview respondents? Do you need help with these interviews? 
Will you schedule focus group or nominal group meetings? Do you need 
help planning and conducting these meetings? 

Using more than one research method is often desirable. As a validity 
check, you can compare data collected from one method with data from 
another method. Do nominal groups of randomly chosen citizens identify 
the same educational needs as a telephone survey of selected community 
leaders? It is especially important to consider several methods when you 
begin a major new program thrust 

Use thepeoplerecaiited to help design theproject,especially if they have 
technical expertise in research methodology. Consult the references at die 
end of this publication for additional assistance with research design ^nd 
sampling procedures. 

Although this step has focused on formal research projects, informal 
reseach requires some planning as well. Decide, for example, whether you 
will rely on informal conversations or systematic reading or both. Flan how 
you will fit this research activity into your schedule to ensure that your best 
intentions are fulfilled. 

Step 7. Develop a Budget. 

By now you probably have an ambitious plan. If on target, you have 
carefully assessed available resources and have not exceeded these limita- 
tions. E^forcprocecdingfurther,however,determinethecostoftheproject. 
Consider all of the direct costs involved with project development, data 
collection, and data analysis. Include printing, postage, telephone, uavc!, 
computing, and personnel expenses. Do not underestimate or cut comers. It 
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is bcuer to face a costly budget now than to find yourself in the middle of 
a project you cannot afford to finish. 

Personnel costs are a critical item that can make orbreak your budget If 
you have to hire outside people for technical assistance or data collection or 
will be c harged back for this assistance within your organization, iheprojec t 
will be expensive. If yourpersonnel services will be contributed, however, 
the project will be much more affordable. 

Have someone critique the budget, just as you would ask for reactions to 
a preliminary research design or draft report Ask if you have underesti- 
mated any items or overlooked other expenses. 

If the budget is too large, you will have to limit the the project in some 
way. You may be able to use only one research method, rather than two or 
three. If you are not sure where to cut the budget right now, go to the next 
step. 

8. Plan How to Use the Data. 

Ask yourself the hard question of how you will actually use all the data 
you plan to collect Chances are, you will have more information than you 
can realistically use to make program decisions. Resist the temptation to 
collect un necessary data just because you would like to know more about the 
respondents. That additional information may very well sit on the shelf 
unused. 

Consider your budget again. If it is more than you can afford, can you 
save money by collecting only essential data to make informed program 
decisions? In the excitement of designing a research project, it is tempting 
to think of all the information you would like to have about your respon- 
dents. But the more data you want, the more timeand money you will spend 
collecting, analyzing, and reporting. 

Before proceeding to collect the data, make any necessary revisions in 
the preliminary plan and budget Be sure this revised plan is affordable and 
calls for collecdng only informau'on that will be used. 

Step 9. Collect and Analyze the Data. 

If you plan a formal research project, establish procedures for data 
collection. Surveys and interviews, for example, require a quesiionnaire or 
interview schedule. Develop this instrument carefully. Generate a list of 
questions you th ink should be asked. Ask those people working with you on 
the project to suggest questions as well. Share the initial list with as many 
people as you can, and ask colleagues in the organization to react to the 
questions. Include people who will helpcollectandanalyzedata.Chooscihe 
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questions that seem most important, revise them as needed, and develop a 
draft of your instrument 

Then test and modify the instrument Ideally, select a small sample of 
people, similar to ihs population chosen for the study, to complete the draft 
instrument Have ihcm give their reactions to it, including questions they 
found confusing and the reasoning they used to answer them. If time and 
money donotpennitextensivepilottesting,atleastrecruitagroupof friends 
and colleagues to work through the sample instrument On the basis of this 
feedback, revise the questions and format of the instrument 

Although not all formal market research methods use questionnaires or 
interview schedules, they do require you to ask questions of respondents. 
Therefore, selecting the right questions, wording them in the right way,and 
presenting them in the right format arc common problems that must be 
solved. For a thorough treatment of developing questions, see Sudman and 
Bradbum (1982). ITiey focus on questionnaires, but their discussion is 
helpful for other uses too. 

For some formal methods, you must also plan how to record data. 
Interviews, for example, present special challenges. Do you want to rely on 
notes that the interviewers take during the interviews? Do you want them to 
refrain from extensive note-taking during the interviews and write detailed 
summaries immediately afterwards? Or do you want to tape-record the 
interviews for later transcription? The same questions apply to group 
discussions. Each technique for recording data has advantages and disad- 
vantages. 

Data analysis requires planning as well. Surveys, for example, typically 
yield quantitative data. The procedures for analyzing quantitative data are 
straightforward, but you must decide how extensive an analysis to do. Just 
as we cautioned earlier against collecting too much data, resist the tempta- 
tion to perform too sophisticated an analysis. Remember that your purpose 
for conducting market research is to make program decisions. Use an 
analysis that enables you to make informed decisions. Simple descriptive 
statistics, for example, frequency distribution, range, and an appropriate 
measure of central tendency, are often sufficient 

Other research methods, such as interviews, focus groups, and informal 
research activities, typically yield at least some qualitative data. Analyzing 
qualitative data is more time consuming a'^d notas straightforward. In some 
cases, you can assign rankings or weights to qualitative responses and then 
do a quantitative analysis. In other cases, such coding loses significant 
information. If this is a problem, develop categories for organizing re- 
sponses and report the data in narrative form. 
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As during project planning and budget development, use colleagues lo 
help you with this phase of the research. Tap people with expertise in 
instrument development and data analysis. If you need additional informa- 
tion, consult the references listed at the end of this publication. 

Step 10. Use the Data to Make Decisions. 

As with the first step, this final step may also seem obvious, but 
remember that the purpose of market research is to make informed judg- 
ments about programs. These judgments do not result automatically from 
the da'u. you have collected. You, the program planner, must decide what 
these data mean for the continuing education programs you conduct. 

Although you are ultimately responsible for these decisions, you do not 
have to make them alone. Involve your planning committee, if there is one, 
and the staff members in your organization who are working on this 
research. Report the findings to these colleagues, using the most appropriate 
format. A written report is the most freguent way of doing so, but it is not 
always the most effective. Busy people may respond better to a succinct 
summary of the major conclusions or a series of charts and graphs. Even an 
informal discussion is sometimes better than a lengthy document to read. 
Include your interpretations and judgments. Then use your colleagues to 
help you decide what program decisions to make on the basis of the data. 
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This brief discussion has offered our perspective on market research in 
continuing education. We have stressed that market research is more than 
needs assessment It includes attention to learning needs, but it also 
considers ieamer motivation to participate in continuing education and the 
resources to do so. Although determining motivation and resources is more 
difficult than assessing needs, we have notedhow some research approaches 
can give some insight into these areas. 

We have not attempted to present a detailed description of every market 
research method. Instead, we have selected a few methods frequently used 
in continuing education, noting their advantages, disadvantages, and key 
points to consider. For further information about these or other market 
research methods, consult the references listed below for further study. 

We have emphasized your importance to market research as program 
planner. Your philosophy of education, personal preferences, and response 
toyour working environment are all strong influences on liowyou do market 
research. We have encouraged you to become more conscious of ho w these 
factors influence you and then consider possible changes in your approach 
to market research. 

We have stressed the importance of ongoing market research. We have 
tried to be realistic, remembering that you are involved in a variety of 
programming responsibilities and may have limited time and money for 
market research projects. We do not want you to conclude that market 
research is an overwhelming UTsk for a program planner. Rather, it is a wa> 
to develop confidence and renewed perspective on program planning. 
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Jossey-Bass. Includes how to analyze available resources, the needs of 
learners, and other providers. 

Coates, J. 1985. Enrollment analysis. Manhattan, Kans.: Learning Re- 
sources Network. Di^usses program user file, geographic distribution 
of participants, use of neighborhood statistics, analyzing catalog distri- 
bution, and marketing applications. 

Falk, C.F., and Miller, P. 1986. A/aa research in adult learning. Manhat- 
tan, Kans.: Learning Resources Network. Outlines an approach to 
systematic market research within a continuing education organization. 

Gilmore, G. D., Campbell , M. D., and Becker, B. L. 1989. Needs assessment 
strategies for health education and health promotion. Indianapolis, 
Ind.: Benchmark Press. Detailed dio^ussion of preparing, conducting, 
and using results of surveys, Delphi, interviews, nominal groups, focus 
groups, and other group strategies. 

Market Research Methods 

Survey 

Converse, J. M. 1986. Survey questions: Handcrafting the standardized 
questionnaire. Beverly Hills, Calif.; Sage Publications. Discusses char- 
acteristics of good questions, research findings on responses to ques- 
tions, and steps involved in pilot testing. 

Dillman, D. A. 1978. Mail and telephone surveys: The total design method. 
New York: Joh. Wiley and Sons. A step-by«step "total design method'' 
of writing questions, constructing questionnaires, and implementing 
mail and telephone surveys. Discusses advantages and disadvantages of 
mail, telephone, and face-io-face surveys. 
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Draves, W. A. 1988. High response surveys. Manhattan, Kans.: Learning 
Resources Networic. Stresses development of short mail surveys for 
market research, including questions to ask, question phrasing, survey 
design, increasing responses, and size of mailing. 

Erdos, P. N. 1983. Professional mail surveys. Malabar, Fla.: Krieger 
Publishing. Discusses conducting surveys from start to finish, with em- 
phasis on designing surveys to enhance responses. 

Fink, A., and Kosecoff, J. 1985. How to conduct surveys: A step-by-step 
guide, Beverly Hills, Calif.: Sage Publications. Includes discussion of 
sampling, survey design, quantitative data analysis, and reporting re- 
sults. 

Fowler, F. J. 1984. Survey research methods, Beverly Hills, Calif.: Sage 
Publications. Discusses sampling, nonrespondents, questionnaire de- 
sign, interviewing, quantitative data analysis, and cUiical issues. 

Frcy, J. H. 1983. Survey research by telephone. Beverly Hills, Calif.: Sage 
Publications. Presents advantages and limitations of telephone surveys, 
questionnaire design, strategies, administration, and costs. Excellent 
comparison of mail, telephone, and facc-to-facc surveys. 

Lockhart, D. C. (ed.). 1984. Making effective use of mailed questionnaires. 
San Francisco: Josscy-Bass. Chapter authors use case studies, research 
results, and theory to address questionnaire meUiodology. 

S udman , S and Bradbum, N. M. 1982. Asking questions: A practical guide 
to questionnaire design. San Francisco. Jossey-Bass. Thorough discus- 
sion of the steps necessary to develop good questions and pilot test 
questionnaires. 

Delphi Technique 

Dclbecq,,A. L., Van de Ven, A. H., and Gustafson, D. H. 1975. Group 
techniques for program planning. A guide to nominal group and Delphi 
processes. Glenview, 111.: Scott, Foresman, and Company. Gives de- 
tailed description and application of nominal group and Delphi tech- 
niques. 

Linstone, H., and Turoff, M. (eds.). 1975. The Delphi method: Techniques 
and applications. New York; Addison -Wesley Publishing. Presents a 
thorough re\ iew of underlying philosophy, applications, research stud- 
ies evaluating the Delphi, specialized techniques, and anticipated future 
uses. 
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Interviewing 

Briggs, C. L. 1986. Learning how to ask: A sociolinguistic appraisal of the 
role of the interview in social science research, Cambridge: Cambridge 
University Press. Advanced discussion of interviewing in field research. 
Includes a chapter on four-phase approach to conducting interviews and 
interpreting data. 

Gorden, R. L. 1987. Interviewing: Strategy, techniqiiesy and tactics. Chi- 
cago: Dorsey Press. Presents an in-depth treatment that includes select- 
ing interviewers, conducting interviews, questioning techniques, and 
case studies. 

Kahn,R.L.,Qn6C3nnd\X''PA951Shedynamicsofintervie\ving:Theory, 
techniques, and cases. New York: Wiley. Old but still useful discussion 
of interviewing theory, principles, and techniques, plus critiques of 
interview transcripts. 

Nominal Group 

Dclbecq, A. L., Van de Ven, A. H., and Guslafson, D. H. 1973. Group 
techniquesfor program planning: A guide to nominal groui) and Delphi 
processes. Glenview, 111.: Scott, Foresman, and Company. Gives de- 
tailed description and application of nominal group and Delphi tech- 
niques. 

Focus Group 

Bellenger, D.N., Bernhardt, K.L., and Goldslucker, J. L. 1916. Qualitative 
research in marketing. Chicago: American Marketing Association. In- 
cludes a chapter on focus group uses and techniques. 

Goldman, A. E., and McDonald, S. S. 1987. The group depth interview: 
Principles and practice. Englewood Cliffs, N J.: Prentice-Hall. Presents 
a thorough discussion of how to conduct effective facus group inter- 
views, including moderator preparation, observation guidelines, data 
analysis, and report writing. 

Krueger, R. A. 1988. Focus groups: A practical g uide for apphed reseat ch. 
Beverly Hills, Calif.: Sage Publications. Applies focus groups to non- 
profit organizations, including characteristics anc uses of focu» groups, 
advantages and limitations, strategies for conducting focus groups, and 
issues to consider. 

Morgan, D. 1988. Focus groups as qualitative research. Beverly Hills, 
Calif.: Sage Publications. Applies focus group to social science setting. 
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Research Skills 

Quantitative Research 

Best, J. W., and Kahn, J. V. 1989. Research in education, Englewood Cliffs, 
NJ.: Prentice-Hall Includes discussion of sampling and data analysis. 

Borg, W., and Gall, M. 1983. Educational research. New York: Longman. 
Includes detailed chapters on sampling and statistical analysis. 

Boyd,H.W.,Weslfall,R.,andSlarch,S.T. mi. Marketing research:Text 
and cases, Homewood, 111.: Irwin. Includes discussion and case ex- 
amples of sampling and data analysis. 

Charles, C. M. 1988. Introduction to educational research. White Plains, 
N.Y.: Longman. Presents an introductory treatment of data collection, 
sampling, and statistical procedures. 

Churchill, G. 1979. Marketing research: Methodological foundations, 
Hinsdale, III: Dryden Press. Gives a clear presentation of research 
design, sampling, data collection, analysis, and reporting. 

Qualitative Research 

Berg, B. L. 1989. Qualitative research methods for the social sciences, 
Boston: Allyn and Bacon. 

Merriam, S. B. 1988. Case study research in education: A qualitative 
approach, San Francisco: Jossey-Bass. Includes discussion of effective 
interviewing, observation, data analysis, and reporting information. 




